Binocular device

ABSTRACT

A binocular device for visualizing optical radiation comprises a support structure, a left camera, and a right camera coupled to the support structure. The left camera comprising left optics and a left image sensor, the right camera comprising right optics and a right image sensor, the left image sensor and the right image sensor being configured to create left and right video signals from detected optical radiation received from the corresponding left and right input optics about a same field of view along respective left and right input optical axes. A specular reflection is detected.

FIELD OF THE INVENTION

The invention relates to a binocular device for visualizing opticalradiation. The invention further relates to a binocular device forvisualizing visible and invisible radiation.

BACKGROUND OF THE INVENTION

When performing a surgical procedure, the surgeon uses bright lights inthe operation room in order to be able to distinguish the tissues to betreated as good as possible. However, not everything can be seen clearlyin that way. For example, certain tissue types, such as certain tumors,are not visible by the human eye. Those tissue types can sometimes bevisualized by a technique of near-infrared imaging. This may be the casefor tumor tissue.

“Binocular Goggle Augmented Imaging and Navigation System providesreal-time fluorescence image guidance for tumor resection and sentinellymph node mapping”, by Suman B. Mondal et al., in Scientific Reports,vol. 5, no. 1, July 2015, discloses a system comprising a near-infrared(NIR) source comprising LEDs and bandpass filters and white flashlightsor surgical light covered with short-pass filters as the white lightsource. An imaging module collects combined color-NIR signal via acustom glass lens. The incoming signal was divided into visible and NIRcomponents by a custom dichroic beam-splitter cube and directed to aseparate color and NIR sensor.

The NIR and color sensors were co-registered. A Windows x64 PC generatessuperimposed color-NIR images, creates a GUI that gives access todisplay, store, and processing functions of image data and duplicatesimages for display on the PC and a head-mounted display modulesimultaneously. The display module consists of a head-mounted display.

SUMMARY OF THE INVENTION

It would be advantageous to provide an improved visualization device. Toaddress this concern, according to an aspect of the invention, abinocular device is provided for visualizing optical radiation,comprising

a support structure;

a left camera and a right camera coupled to the support structure, theleft camera comprising left optics and a left image sensor, the rightcamera comprising right optics and a right image sensor, the left imagesensor and the right image sensor being configured to create left andright video signals from detected optical radiation received from thecorresponding left and right input optics about a same field of viewalong respective left and right input optical axes;

the device further comprising a processing unit configured to:

receive a signal representing a left image from the left camera and aright image from the right camera, the left image and the right imagebeing captured substantially simultaneously by the left camera and theright camera, respectively;

compare the left image to the right image; and

detect a specular reflection based on a result of the comparison.

According to another aspect, a method of visualizing optic radiation isprovided, the method comprising

receiving radiation, about a same field of view along respective leftand right input optical axes, by left and a right input optics coupledto a support structure, and transmitting the light onto a left imagesensor of a left camera and a right image sensor of a right camera,respectively, the left camera and the right camera being coupled to thesupport structure; and

creating left and right video signals from the detected radiationreceived by the left camera and the right camera, respectively;

receiving, by a processor, a signal representing a left image from theleft camera and a right image from the right camera, the left image andthe right image being captured substantially simultaneously by the leftcamera and the right camera, respectively;

comparing, by the processor, the left image to the right image; and

detecting, by the processor, a specular reflection based on a result ofthe comparison.

According to another aspect, a binocular device is provided forvisualizing optical radiation, the device comprising

a support structure;

a left camera and a right camera coupled to the support structure, theleft camera comprising left optics and a left image sensor, the rightcamera comprising right optics and a right image sensor, the left imagesensor and the right image sensor being configured to create left andright video signals from detected optical radiation received from thecorresponding left and right input optics about a same field of viewalong respective left and right input optical axes;

a left display and a right display coupled to the support structure andarranged to be viewed by a pair of eyes of a user through a lefteyepiece operatively connected to the left display and a right eyepieceoperatively connected to the right display, wherein the left display andthe right display are configured to present left and right video images,formed with visible light by the left display and the right display,based respectively on the left and right video signals;

wherein the binocular device is configured to, in a particularvisualization mode, alternatingly show a left image based on the leftvideo signals on the left display and a right image based on the rightvideo signals on the right display.

This allows a user to distinguish a specular reflection.

According to another aspect, a method of visualizing optic radiation isprovided, the method comprising

receiving radiation, about a same field of view along respective leftand right input optical axes, by left and a right input optics coupledto a support structure, and transmitting the light onto a left imagesensor of a left camera and a right image sensor of a right camera,respectively, the left camera and the right camera being coupled to thesupport structure; and

creating left and right video signals from the detected radiationreceived by the left camera and the right camera, respectively;

presenting, by a left display and a right display coupled to the supportstructure and to be viewed by a pair of eyes of a user through a lefteyepiece operatively connected to the left display and a right eyepieceoperatively connected to the right display, left and right video imagesformed with visible light based respectively on the left and right videosignals,

wherein the presenting comprises, in a particular visualization mode,alternatingly showing a left image based on the left video signals onthe left display and a right image based on the right video signals onthe right display.

This allows a user to distinguish a specular reflection.

It would be advantageous to provide an improved visualization device. Toaddress this concern, according to an aspect of the invention, abinocular device is provided for visualizing visible and invisibleradiation. The binocular device comprises

a support structure;

a left camera and a right camera coupled to the support structure, theleft camera comprising left optics and a left image sensor, the rightcamera comprising right optics and a right image sensor;

the left image sensor and the right image sensor being configured tocreate left and right video signals from detected optical radiationreceived from the corresponding left and right input optics about a samefield of view along respective left and right input optical axes,

at least one of the cameras being sensitive to both radiation in aninvisible wavelength band of radiation and radiation in a visible lightwavelength band of radiation, the input optics of said at least one ofthe cameras being transmissive for the invisible wavelength band andreductive for the visible light wavelength band.

The combination of features may help to present a stereoscopicallyrealistic depiction of features in the visible light wavelength rangeand the invisible wavelength range. The input optics that istransmissive for the invisible wavelength band and reductive for thevisible light wavelength band may help to improve image quality. This isbased on the notion that the intensity of received invisible radiation,such as infrared radiation, is in most cases much less than theintensity of visible light. In many practical situations, the visiblelight is abundantly present, while the intensity of the invisiblewavelength band of radiation is much less. The reduction of the visiblelight wavelength band of radiation, without reducing the invisiblewavelength band of radiation too much, brings the intensity levels ofboth wavelength bands of radiation closer together. This may improveimage quality, in particular in combination with low-cost and/orlight-weight optics and image sensors.

For example, the infrared may be near-infrared (NIR). In applications ofviewing diffuse-reflecting or fluoroscopic radiation in thenear-infrared wavelength band of radiation, the intensity of thereceived relevant near-infrared radiation will in many cases be muchless than the received radiation in the visible wavelength band.

The binocular device may further comprise a left display and a rightdisplay coupled to the support structure and arranged to be viewed by apair of eyes of a user through a left eyepiece operatively connected tothe left display and a right eyepiece operatively connected to the rightdisplay, wherein the left display and the right display are configuredto present left and right video images, formed with visible light by theleft display and the right display, based respectively on the left andright video signals. This allows to create a display device withbuilt-in cameras and displays that provide improved visualization ofradiation in the invisible wavelength band, for example in ahead-mounted device or a hand-held device.

The input optics of the camera that is sensitive to radiation in theinvisible wavelength band may comprise a polarizing filter comprising atleast one layer of a polarizing material. A polarizing filter having thedesired properties may be made up of particularly light-weight andcost-effective material. For example, the material used in manysunglasses reduces the intensity of visible light considerably whilebeing largely transmissive to certain invisible wavelength bands ofradiation, such as near-infrared and infrared.

The polarizing filter may comprise at least two layers of the polarizingmaterial, having a mutually orthogonal polarization direction. This way,about 98% to 99% of the visible light may be blocked, in a relativelyspectrally linear fashion. Moreover, about 90% to 95% of certaininvisible wavelengths of radiation, such as a near-infrared wavelengthrange, may be transmitted through the polarizing filter.

The binocular device may further comprise a light source coupled to thesupport structure, capable of generating radiation within at least theinvisible wavelength band and the visible light wavelength band.

Preferably, the light source is configured to generate beams of emittedvisible light and invisible radiation that are aligned, for example bymeans of optical elements, to be substantially identical in geometricalshape and position. This way, the detected image may be more consistent.

The light source may comprise a polarizing filter configured to polarizethe visible light within the visible light wavelength band output by thelight source and transmit the radiation in the invisible wavelengthband, wherein a polarization direction of the polarizing filter of thelight source is substantially orthogonal to a polarization direction ofthe polarizing filter of the input optics. This is another way to reducethe amount of visible light significantly while keeping most of theinvisible radiation.

The input optics corresponding to the camera that is sensitive toinvisible radiation may comprise a diaphragm having an aperture, thediaphragm around the aperture being reductive for light in the visiblelight wavelength band, while the diaphragm is transmissive for the lightin the infrared wavelength band. This allows to apply a diaphragmselectively to the visible light wavelength band while allowing theradiation in the invisible wavelength band to pass, substantiallywithout being affected by the diaphragm. In addition to reducing theintensity of the visible light compared to the intensity of theinvisible radiation, this feature allows to make improved use of theoptics. Conventional lenses are known to have different focal spots fordifferent wavelengths of radiation, due to dispersion. To enable optimalfocus for each wavelength band of radiation (e.g., red, green, blue, andinfrared), complex optics is necessary, for example by separating eachwavelength band in a separate bundle and separately focusing each bundleusing separate optics. Using a diaphragm with a relatively smallaperture, the depth of focus is increased, which reduces this problembut also reduces the intensity of the radiation. Considering that theintensity of the visible light is much higher than the intensity of theradiation in the invisible wavelength band, the diaphragm as set forthherein has the advantages of a diaphragm for the visible light withoutreducing the low-intensity radiation in the invisible wavelength band.The depth of focus is increased for the visible light wavelength band ofradiation, providing a sharp image of the visible light. This allows tooptimize the focus of the input optics for the invisible wavelength bandof radiation. Thereby, the input optics can be simplified because theinput optics do not have to take into account the dispersion.

The input optics may comprise a lens with autofocus, wherein theautofocus is configured to bring into focus the radiation in theinvisible wavelength band. This can be done by using a known autofocusfunctionality. This way, each wavelength band that is recorded by thecamera may be made into a sharp image.

The input optics of the infrared sensitive camera may comprise anadditional filter that is reductive for the light in the visible lightwavelength band in addition to the diaphragm. To further reduce thevisible light intensity, an additional filter for the visible light maybe added, wherein the additional filter does not have an aperture.

The input optics of the camera that is sensitive to radiation of theinvisible wavelength band may comprise a filter comprising iodine forselectively reducing the radiation in the visible light wavelength band.Iodine is known to reduce such radiation, while being transmissive forcertain invisible wavelength bands, such as an infrared or near-infraredwavelength band.

The binocular device may comprise a processing unit configured toreceive a signal representing a left image from the left camera and aright image from the right camera, the left image and the right imagebeing captured substantially simultaneously by the left camera and theright camera, respectively, compare the left image to the right image,and detect a specular reflection based on a result of the comparison.This is convenient for detecting specular reflections. It is observedthat the processing unit does not have to be fixed to the supportstructure. However, the processing unit may have a communicationconnection (wired or wireless) for exchange of the video signals withthe image sensors and displays. The left image and the right image maybe captured substantially simultaneously while the light sourceconnected to the support structure is switched on to emit the visiblelight and the radiation in the invisible wavelength band. This way, thereflections may have a more predictable appearance.

The binocular device may further comprise a light source coupled to thesupport structure, for generating at least infrared light and visiblelight, wherein the light source is configured to intermittently emit theinfrared light while keeping the visible light intensity substantiallyconstant, wherein the camera is configured to capture at least one imagewith the emitted infrared light and at least one image without theemitted infrared light. This provides improved quality images, becausethe visual light images do not suffer from possible deterioration causedby the emitted radiation in the invisible wavelength range. Moreover, noflicker in the visible wavelength range may be caused. Moreover, thismay allow to generate improved quality images by combining the imagecaptured with the emitted infrared light with the image captured withoutthe emitted infrared light.

The processing unit may be configured to calculate an enhanced infraredimage based on the captured image with the emitted infrared light andthe captured image without the emitted infrared light.

Each of the left camera and the right camera may be sensitive toradiation in the invisible wavelength band and radiation in the visiblelight wavelength band, while each of the left input optics and the rightinput optics may be transmissive for the infrared wavelength band andreductive for the visible light wavelength band. This way, the radiationin the invisible wavelength band may be made visible stereoscopically.

The image sensor of the camera that is sensitive to the radiation in theinvisible wavelength band may comprise a sensor die that is sensitive toboth the infrared wavelength band of radiation and the visiblewavelength band of radiation, wherein the sensor die may be configuredto output the video signal corresponding to both the radiation in theinfrared wavelength band and the radiation in the visible wavelengthband. This allows a relatively simple and light-weight design of thebinocular device. Moreover, in combination with the reduction of thevisible light the image quality can still be high.

According to another aspect of the invention, a method of visualizingvisible and invisible radiation is provided. The method comprises

receiving radiation, about a same field of view along respective leftand right input optical axes, by left and a right input optics coupledto a support structure, and transmitting the light onto a left imagesensor of a left camera and a right image sensor of a right camera,respectively, the left camera and the right camera being coupled to thesupport structure,

wherein at least one of the cameras is sensitive to both radiation in aninvisible wavelength band of radiation and radiation in a visible lightwavelength band of radiation, wherein the input optics of the camerathat is sensitive to the invisible wavelength band of radiation istransmissive for the invisible wavelength band of radiation andreductive for the visible light wavelength band of radiation;

creating left and right video signals from the detected radiationreceived by the left camera and the right camera, respectively;

presenting, by a left display and a right display coupled to the supportstructure and to be viewed by a pair of eyes of a user through a lefteyepiece operatively connected to the left display and a right eyepieceoperatively connected to the right display, left and right video imagesformed with visible light, based respectively on the left and rightvideo signals.

The person skilled in the art will understand that the featuresdescribed above may be combined in any way deemed useful. Moreover,modifications and variations described in respect of the system maylikewise be applied to the method and to the computer program product,and modifications and variations described in respect of the method maylikewise be applied to the system and to the computer program product.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following, aspects of the invention will be elucidated by meansof examples, with reference to the drawings. The drawings arediagrammatic and may not be drawn to scale. Throughout the drawings,similar items may be indicated with the same reference numerals.

FIG. 1 shows a diagram of a system for combining visual and infraredimaging.

FIG. 2A shows a schematic view of a camera of a head-mounted device.

FIG. 2B shows a schematic view of a camera of a head-mounted deviceincluding a pair of orthogonally oriented polarizers.

FIG. 3A illustrates a pair of orthogonally oriented polarizationfilters.

FIG. 3B illustrates a pair of diaphragms, made of orthogonally orientedpolarization filters having an aperture.

FIG. 3C illustrates an effect of a diaphragm filter.

FIG. 4 illustrates an optical principle of stereoscopic viewing using abinocular device.

FIG. 5 shows a first example of a specular reflection.

FIG. 6 shows a second example of a specular reflection.

FIG. 7 shows a third example of a specular reflection.

FIG. 8 shows a diagram of a combined camera and viewer for one eye.

FIG. 9 shows an example of a timing diagram of the binocular device inoperation.

FIG. 10 shows a partly worked open view of an exemplary head-mountablebinocular device, viewed from below.

FIG. 11 shows a partly worked open view of the same head-mountablebinocular device, viewed from aside.

FIG. 12 shows a head-mountable device attached to the head of a user.

FIG. 13 shows several wavelength bands.

FIG. 14 shows a flowchart of a method of visualizing visible andinvisible radiation.

DETAILED DESCRIPTION OF EMBODIMENTS

Certain exemplary embodiments will be described in greater detail, withreference to the accompanying drawings.

The matters disclosed in the description, such as detailed constructionand elements, are provided to assist in a comprehensive understanding ofthe exemplary embodiments. Accordingly, it is apparent that theexemplary embodiments can be carried out without those specificallydefined matters. Also, well-known operations or structures are notdescribed in detail, since they would obscure the description withunnecessary detail.

FIG. 1 shows an overview of an embodiment of a system for combiningvisual and infrared imaging. Although the system is explained here inthe context of infrared imaging, a similar arrangement may be made fornear-infrared or ultraviolet, or any other invisible wavelength band ofradiation. The system comprises a head-mounted display 101 with a pairof built-in cameras. For example, the system is capable of magnificationof the viewed pictures using for example a scale factor between 1 and 5.Recording can continue based on the original images while the user viewsmagnified images. Full HD view with a wide-angle of 40 degreeshorizontal may be provided. The system may be implemented to be wearableand autonomous. For example, the head-mounted display 101 with camerasmay be provided with additional computational power by means of awearable PC that may be carried by the user and that may be connected tothe head-mounted display 101 by means of a wired (or a wireless)interface. The wearable PC and the head-mounted display 101 with camerasmay be battery powered and communicate with external devices through awireless (or a wired) connection 107. This provides the user withfreedom of movement. The head mounted display 101 may have a low weightof e.g. less than 250 grams. The system may allow for 3d recording, forexample through direct image streaming and recording, and may allowdirect viewing of the image stream by the operating team on a largerdisplay screen 105. The system may further comprise an external userinterface device 104 that can be operated by the assistant parallel tothe user of the head-mountable device 101. The user interface device 104may be connected wired or wirelessly to the wearable PC 102 and/ordirectly to the head-mounted display 101. The system may furthercomprise a remote control by means of a foot pedal 106 or tablet, whichmay be connected wired or wirelessly to the wearable PC 102 and/ordirectly to the head-mounted display 101. The system may further beconfigured to provide augmented reality picture in picture overlay ofpre-operative images on demand.

For example, the point of view follows the head movements of thesurgeon.

Existing systems are bulky, heavy, not head-wearable, not well alignedwith the eye, delayed in visualization of the processed image and notautonomous as well as having a fixed focus distance of >2 meters.

Most surgical disciplines (open or laparoscopic) such as generalsurgery, oncology plastic surgery, urology, gynecology,otorhinolaryngology, thorax surgery, and neurosurgery have one thing incommon: the necessity to correctly identify and prevent damage to vitalanatomical structures that need to be preserved (e.g. nerves, lymphatictissue and blood vessels) and to identify the targeted tissue that needsto be removed or treated. This is challenging, especially whenconsidering natural anatomical variations between individuals (the exactposition of vital structures varies per individual). Damaging vitalstructures can cause severe surgical complications, such as vascularinjury, ureter lesion, bile duct injury and nerve damage. Suchcomplications have huge impact on patients and the healthcare system.Therefore, a solution that can reduce these risks is of great importancefor surgeons, their patients and society as a whole.

Recognition of critical non visible anatomical structures (tissues) atworking distance is desirable and may be provided by the techniquesdisclosed herein.

Real-time visualization (or e.g. visualization latency max 30 ms) may beprovided in a head mounted 3D display 101, to be able to perform surgerywith optimal spatial perception of depth and bodily orientation forextremely refined microscopic work. Longer delays in visualization areknown to cause disorientation and nausea (motion sickness) when the headis freely moving, in case visual input does not match with bodilyproprioception input.

Contactless recognition of critical non-visible anatomical structures(tissues) throughout normal working visual distance range (e.g. 30-50cm) may be provided. Real-time, or very low-latency, (max 30 ms)simultaneous visual (VIS) and near-infrared (NIR) images in 3D, foraugmented reality visualization with optimal spatial depth perception,may be produced. Computer analytics embedded algorithms (for example,implemented on FPGA) within the wearable computer system 102 or withinthe head mountable device 101 may allow to recognize (in real-time)spectral fingerprints (characteristics) of different tissue types andhuman organs, whilst producing an enhanced AR overlay of the criticaltissues from data invisible to the human eye on the actual clinicalfield of view (FOV) by pixel alignment of two data streams. For example,visible light-based video stream and a video stream corresponding to anon-visible wavelength range, such as near-infrared streams for each eyeto visualize in one 3D HD video stream. The system may be built around acompact lightweight (e.g. less than 250 grams), head mountable device101 for optimal ergonomics and long term usage (up to e.g. 4-6 hours)without neck strain or eye fatigue. The system may further comprise anautonomous compact wearable computer system 102 (for example, batterypowered with wireless 107 high-speed data and video transport to anexternal server), for optimal freedom of movement and minimum lag.

The system may be able to discern between shiny light spots caused byreflectance on the tissue versus light emission that truly originatesfrom the tissue, by comparing the simultaneously acquired Left and Rightimages. The position of reflected shiny spots may differ between theLeft and Right image, because of the difference in reflection angletowards the light source. The position of a light spot that is emittedby the tissue itself (e.g. due to fluorescence) or that originates frombodily contrast may however remain on the same position in a convergedimage. In particular where a weak signal is visualized from an addedfluorescent marker or (even weaker) from bodily autofluorescence, theselective suppression of shiny spots from tissue reflection may beadvantageous. This suppression method for shiny reflectance spots can becombined with other methods, such as crossed polarization between lightsource and camera input.

By locally processing within an FPGA, data transfer speed limitationsdue to longer distances may be avoided. The head-mountable device 101may allow real-time recognition of critical non-visible tissues based oncomputer analytics embedded unique algorithms (FPGA) to recognizespectral signatures in real-time.

Dedicated image processing embedded software (e.g. implemented on FPGA)may cater for various display technologies (e.g. OLED, FLCoS & AMLCD) tobe displayed on a single hardware platform and convert industry standardvideo signals to the necessary video format and control signal for themicro display. 3D Augmented Reality contrast overlay of critical tissuesmay be provided on the RGB visible video stream, to be able to performsurgery with optimal spatial depth perception. Dedicated software maymake this possible. A head mountable autonomous system with ergonomicadvantages and freedom of movement may be provided, preferably withoutwires connecting it to an external source.

The Head-mountable device 101 may be tethered 103 to a wearable computer102. This may facilitate 3D recording of procedure for patient files andeducational purposes, 2D play-back with real-time algorithm calculationfor convergence compensation, 2D or 3D real-time (wireless 107)streaming to nearby screens 104, 105 for improved optimal teamwork, butalso for remote assistance, medical patient data recording and/oreducational purposes, intelligent zoom function smoothly between 1 to5-10-20 times for optimal 3D accuracy, software adjustable 3Dconvergence based on working distance, cameras angle positioning from 0to 20 degrees downwards, for optimal ergonomics (minimize neck hernia)and so that the image may be captured at the macula on the retina of theeye for optimal sharpness. The system may be used in carrying outresearch regarding multiple new medical application areas for tissuestructure recognition (oncology, dermatology, etc.). The system can beapplied to other non-medical application areas like Forensic research.Videos or information to the user (remote support) can be made to pop-upinto the view of the user when needed (e.g. voice controlled, gesture orvia foot pedal). The Optical brightness can automatically adapt toenvironmental lighting conditions for ideal viewing conditions,providing ambient through the lens light Sensing technique. Embedded IMUmay register head movement of the user in 3 axes (e.g. Yaw, Pitch &Roll) in real-time in a virtual (computer generated) environment. Thismay allow for panoramic 3D-stitching of multiple stereoscopic imagesfrom multiple perspectives. A wireless 107 footswitch 106 and/orvoice/gesture control of various functions may be provided.

The device may operate separately controllable sources for the visible(VIS) or UV, or near-infrared (NIR) spectral ranges. Both bundles may becarefully aligned. Automatic compensation for traces of near-infrared(NIR) light present within the ambient light (e.g. as with fluorescenttubes) by modulation of the NIR-lighting to obtain consecutive NIRimages with NIR source on and off, thus allowing for dark-fieldcompensation. All this without a need to modulate the visual lightsource (which might cause nuisance flicker, or even possibly triggerepileptic seizures).

Illumination of the anatomic structures may be performed in at least twospectral ranges: Namely the visible light and the invisible UV ornear-infrared region. Lighting may be done under well-defined lightconditions on light Temperature (K), Color Rendering Index (CRI), angleand beam shape. However, the system may be configured to automaticallyadapt to differing light conditions.

Two multi-spectral cameras (each combining the visual range by RGB videowith separate UV or near-infrared spectral visualization), may beprovided. A wearable computer 102 with data processing, embeddedalgorithms (FPGA) to recognize (real-time) spectral fingerprints ofdifferent tissue types and human organs may be provided. The user may beable to choose between various tissue types for visual augmentation.Optimal depth perception may be supported by 3D software real-timecorrection of parallax and convergence. Optimized data transporttechnology (via USB 3.2 Gen 2x2) for fast data exchange up to 40 GB/sdata rate may be provided. A single cable may be implemented to controltwo NIR cameras and two micro-displays. Data processing of the stream ofimage pairs may be done externally via a tethered 103 or wirelesslyconnected processor in wearable computer 102 that may feed the outputimage to the Head Mounted Display 101 within, for example, less than 30milliseconds. Enhanced AR overlay of critical tissue contrasts thatotherwise would be invisible to the human eye may be displayed on theactual clinical field of view (FOV) by pixel alignment of two datastreams to view in one 3D image. The system may cater for variousdisplay technologies (OLED, FLCoS & AMLCD) and convert industry standardvideo signals to the necessary data and control signal for the microdisplay. The system may be controlled, for example, by foot pedal, voicecontrol or remote tablet touch screen.

FIG. 2A shows a schematic view of a camera of a head-mounted device 101.Two cameras may be implemented, one for each eye, to support stereoviewing. The camera may comprise a housing with a wall 208 thatoptically isolates the optic components from outside radiation, exceptfor the opening 209. The camera may comprise input optics 210 and animage sensor 206. The input optics 210 may comprise, from the opening209 arranged in sequence towards the image sensor 206, a first lens 201,a notch filter 202, a visible light reduction filter 203, a visiblelight diaphragm 204, and a second lens 205. The order of thesecomponents may vary for different implementations, and the number andconfiguration of the lenses 201, 205 may be different. One or more ofthe lenses 201, 205 may be movable for purpose of focusing the imagethat is sensed by the image sensor 206. The image sensor 206 maycomprise a silicon implemented sensor with coatings to make the imagesensor 206 sensitive to both at least one wavelength of visibleradiation, such as a red wavelength range, a green wavelength range, anda blue wavelength range, and at least one wavelength range ofnon-visible radiation, such as a near-infrared wavelength range, aninfrared wavelength range, and/or an ultraviolet wavelength range. Theoptional notch filter 202 may at least partially suppress excitation ofa certain undesired wavelength range.

The visible light reduction filter 203 may be adapted to suppress, orreduce, a large portion of visible light wavelengths. Moreover, thevisible light reduction filter 203 may be adapted to allow to pass, orbe transmissive for, light of a non-visible wavelength range, for whichnon-visible wavelength range the image sensor 206 is sensitive. Forexample, the visible light reduction filter 203 may be made of adichroic material, which reflects or absorbs most of the light in thevisible light wavelength range while transmitting most of the light inthe relevant non-visible wavelength range. The visible light reductionfilter 203 is adapted to allow to pass a portion of the visible light.

Examples of a suitable material for visible light reduction filter 203and/or visible light diaphragm 204 include H-sheet Polaroid, which is apolyvinyl alcohol (PVA) polymer impregnated with iodine, and K-sheetPolaroid, which comprises aligned polyvinylidene chains in a PVA polymercreated by dehydrating PVA. Another example material is a coatablepolymer polarizer formed with a composition that includes a rigidrod-like polymer capable of forming a liquid crystal phase in a solvent,wherein the rigid rod-like polymer may form an achromatic polarizer, asdisclosed in US 2016/0266292 A1. Yet another example material is apolarizing plate including a stretched laminate that is a laminateincluding a base material layer and a hydrophilic polymer layer and hasundergone a stretching process, and at least a dichroic substance isadsorbed to the hydrophilic polymer layer, as disclosed in U.S. Pat. No.8,559,105 B2. Another example is based on material usually used indielectric beam splitters. Since it is possible to specify thewavelength at which the radiation is split, it is possible tomanufacture a dielectric coating that is transparent for an invisiblewavelength band of radiation, such as NIR light, and reflective for avisible light wavelength band of radiation. In this regard, it may beuseful to fix the visible light filter 203 or visible light diaphragm204 made up of a dielectric beam splitter at an angle (of e.g. 45degrees) with respect to the optical axis of the camera (notillustrated).

The visible light diaphragm 204 is a diaphragm, which may have a fixedaperture or a variable aperture. The diaphragm is made of a materialthat reduces (or even completely blocks) the light in the visible lightwavelength range, while being transmissive for the relevant non-visiblewavelength range, such as near-infrared radiation. For example, thevisible light diaphragm 204 may be made of a dichroic material thatreflects or absorbs most of the light in the visible light wavelengthrange while transmitting most of the light in the relevant non-visiblewavelength range. For example, the visible light diaphragm may be madeof the same material as the visible light reduction filter 203, or adifferent material. The visible light diaphragm 204 may be fullynon-transmissive for the light in the visible light wavelength range(except for the light passing through its aperture). Alternatively, thematerial of the visible light diaphragm 203 may allow to pass a(relatively small) portion of the visible light.

The lens 205 cooperates with the lens 201 to create a bundle of incominglight from the opening 209 onto the image sensor 206. The two lenses maybe movable with respect to each other, in order to provide a focusfunction. The lenses may be transmissive for both visible light and therelevant non-visible wavelength range.

It will be noted that, although FIG. 2A shows both the visible lightreduction filter 203 and the visible light diaphragm 204, this is not alimitation. In certain embodiments, either one of these components maybe provided. In both cases the input optics 210 is transmissive for therelevant non-visible wavelength band and reductive for a visible lightwavelength band. As stated above, the input optics does not fully blockthe visible light, but merely reduces it to a level that is suitable forthe image sensor 206. Moreover, the input optics do not have to allow100% of the light in the relevant invisible wavelength range to pass.Due to, for example, restrictions in available materials, the inputoptics does reduce the light in the relevant invisible wavelength rangea bit, too.

The input optics 210 is reductive for radiation in the visible lightwavelength band. For example, the intensity of radiation in the visiblelight wavelength band may be reduced by at least 75%, preferably by atleast 90%, more preferably by at least 95%, even more preferably by atleast 98%. The input optics is transmissive for radiation in the choseninvisible wavelength band. For example, the input optics may transmit atleast 80% of the intensity of received radiation in the invisiblewavelength band of radiation, preferably at least 90%, more preferablyat least 95%. For example, reduction of visible light is at least 75%and transmission of invisible radiation is at least 80%. For example,reduction of visible light is at least 95% and transmission of invisibleradiation is at least 90%.

FIG. 2B shows the camera of FIG. 2A, in which the visible lightreduction filter 203 has been implemented as a pair of orthogonallyoriented polarization filters 203 a and 203 b. Moreover, the visiblelight diaphragm 204 has been implemented as a pair of diaphragms 204 a,204 b that are made of an orthogonally oriented polarization filtermaterial. For example, such a material comprises an iodine-containingpolymer. Such a pair of orthogonally oriented polarization filters isknown to transmit about 90% to 95% of the radiation in the near-infraredwavelength range, while removing about 98% to 99% of the visible lightin a spectrally linear fashion. It is noted that such aniodine-containing polymer does not remove the visible light entirely.This is in line with the purpose of the filter and diaphragm to remove alot of the visible light but not all.

FIG. 3A illustrates the pair of orthogonally oriented polarizationfilters 203 a and 203 b in a see-through direction. The diagonal linesindicate a polarization direction. It is noted that the polarizationdirection of the first filter 203 a is orthogonal to the polarizationdirection of the second filter 203 b. It is observed that in certainapplications it may be sufficient to provide only one polarizationfilter instead of two. Moreover, in case two polarization filters areprovided, the amount of visible light reduction may be made variable bymaking the filters 203 a and 203 b rotatable with respect to each otherunder control of a control software, for example.

FIG. 3B illustrates the pair of diaphragms 204 a, 204 b, made oforthogonally oriented polarization filters having an aperture 207. Thediagonal lines indicate a polarization direction. It is noted that thepolarization direction of the first aperture filter 204 a is orthogonalto the polarization direction of the second aperture filter 204 b. It isobserved that in certain applications it may be sufficient to provideonly one polarization filter-based diaphragm instead of two. Moreover,in case two polarization filter-based diaphragms are provided, theamount of visible light reduction may be made variable by making theaperture filters 204 a and 204 b rotatable with respect to each otherunder control of a control software, for example.

Alternatively, certain implementations may comprise one polarizingfilter 203 a and one polarizing diaphragm filter 204 a, omitting eitherone or both of the second filter 203 b and second diaphragm filter 204b. The polarization direction of the one filter 203 a may be orthogonal(or have another desired orientation or be variable) with respect to theone diaphragm 204 a.

FIG. 3C illustrates an effect of the diaphragm filter 204. The graphshown in FIG. 3C has the wavelength on the vertical axis and thedistance from the camera on the horizontal axis (in an arbitrary scale).Depth of field is the distance between the nearest and the farthestobjects, as seen from the camera, that are in acceptably sharp focus inan image. Since the visible light can only pass through the aperture 207of the diaphragm filter 204, the visible light has a relatively largedepth of field, as indicated by the large space in between the arrows inthe horizontal line representing a visible wavelength (VIS). However,the invisible light wavelength range that is allowed to pass thematerial of the diaphragm filter 204, has a relatively small depth offield, as illustrated by the smaller space in between the arrows in thehorizontal line representing a near-infrared wavelength (NIR). Since theinvisible light may have much less intensity than the visible light, itmay be advantageous to keep all the invisible light, while reducing theamount of visible light via the diaphragm filter 204. Moreover, due todispersion, the focal spot of a lens in respect of different wavelengthsdiffers. To create well-focused image in all detected wavelengths, anexpensive and heavy lens system would normally be necessary. The presentdiaphragm filter creates a large depth of field for the visible lightwavelength range, while creating a narrow field of depth for theinvisible light wavelength range. This principle may be exploited byoptimizing the focus of the lens for the invisible light with the narrowfield of depth. This way, the invisible light image is sharp andwell-focused. Since the visible light has a large field of depth, thevisible light image is also sharp and well-focused. Thus, the systemallows a light-weight, high-quality imaging device with a relativelysimple lens and filter.

FIG. 4 illustrates an optical principle of stereoscopic viewing. Itshows a binocular device 400. It is observed that the figure issimplified in order to explain the concepts disclosed herein. In manypractical embodiments, where the focal point may be further away, forexample at a working distance of about 20 centimeters to 1 meter, theline of vision of the left eye 413 may be almost parallel to the line ofvision of the right eye 423. So the left optics 410 may be implementedwith a very small angle or parallel to the right optics 420.

The binocular device 400 comprises a support structure 404 on which thecomponents of the binocular device 400 are attached. In certainembodiments, the whole device may be encapsulated in a housing.Alternatively, the components of the left image may be encapsulated in afirst housing and the components of the right image may be encapsulatedin a second housing, both housings being attached to the supportstructure 404. In either way, the left optics 410 and the right optics420 are fixed with respect to each other. In certain embodiments theleft optics 410 and the right optics 420 may be movable with respect toeach other, for example for customizing the device 400 for a particularuser.

The binocular device 400 may comprise left optics 410. The left optics410 may comprise a left camera 411 and a left display 412. The rightoptics 420 may comprise a right camera 421 and a right display 422. Theleft camera 411 and right camera 421 may comprise the visible lightreduction filter 203 and/or visible light diaphragm 204, as explainedabove. However, certain embodiments may omit such a filter or diaphragmin one or both of the cameras.

The binocular device 400 may further comprise a light source 403. Thelight source 403 may be any device that can emit light in a desiredwavelength range. Examples of suitable light sources include lightemitting diode (LED), incandescent light, halogen light, or anotherlight source. The light source 403 may comprise two or more lightsources that generate light of different spectra (for example, a lightsource that generates white visible light and a light source thatprimarily generates light of an invisible wavelength range, such as anear-infrared wavelength range), and optics to combine the lightproduced by these light sources into one bundle. Alternatively, thelight source may comprise a single emitter that emits light in bothvisible and invisible wavelengths. Yet alternatively, the light source403 may substantially emit only light of the invisible wavelength range.For example, in certain applications it may be assumed that visiblelight is available in abundance by environmental lighting generated byoperation room lights or sunlight, so that no additional visible lightneeds to be generated by the binocular device 400.

As illustrated in the figure, in operation, the light source 403generates light. For example, light ray 405 may hit an object 401, suchas a tissue to be inspected. This is illustrated by arrow 405. Thetangent plane of the tissue at the point of incidence 408 of the lightray 405 with the tissue is illustrated by dotted line 402. In case of adiffuse reflection or fluorescence, for example, light travels from thepoint of incidence 408 to both the left camera 411, as illustrated byarrow 406, and to the right camera 421, as illustrated by arrow 407.

FIG. 5 shows the binocular device 400 in case of a specular reflectionby tissue 501 at point of incidence 508 with tangent plane 502. Thelight ray 505 emitted by the light source 403 is reflected mostlytowards, in the present example, the right camera 421 along ray 507.Only little to none of the light is reflected to the left camera 411, asindicated by dotted line 506.

FIG. 6 shows that, of course, there is not just one light ray but alight bundle 605 that is reflected in a bundle 607 into the right camera421. The geometrical beam shape generated by the light source 403 inrespect of the visible wavelength range may be, as much as possible,identical to geometrical beam shape generated by the light source 403 inrespect of the non-visible wavelength range.

FIG. 7 illustrates another example of specular reflection, wherein apart 707 of the light bundle 705 emitted by the light source 403 isreflected by the tissue 701 towards the right camera 421 and anotherpart 706 of the light bundle 705 is reflected towards the left camera411.

It would be undesirable if specular reflections would be mistakenlyrecognized by the user as actual diffusely-reflecting contrast orfluorescence originating from tissues. In the case that both camerasreceive off-axis specular reflections, the output image could bemistakenly recognized as, for example, NIR-induced fluorescence from twoseparate spots (X & Y in FIG. 7) on the tissue surface. The VIS lightsource may be kept continuously on (to avoid visible flicker). Thedichroic diaphragm and/or filter may suppress specular reflections inthe visible wavelength much stronger than NIR ones. The NIR source canbe modulated ON/OF without causing visible flickering. Images may becaptured by the cameras 411 and 421 while the NIR light source is on andwhile the NIR source is off. The resulting pairs of images (one pair(left/right) with the NIR light source on and one pair with the NIRlight source off) may be processed to enhance the NIR visualization. Forexample, the images obtained with the NIR light source off may besubtracted from the images obtained with the NIR light source on. Thisway, an image with enhanced visualization of the NIR is created. The NIRimage may be converted to a visible color and blended into the imagethat was obtained with the NIR light source off, so that a combinedvisualization of the visible light and the NIR light is created. Thiscombined visualization may be displayed on the left display 412 andright display 422. However, it is also possible to display thevisible-light only images on the left display 412 and right display 422,without visualization of the NIR light.

As a special imaging mode, the visualization of the NIR light in theleft and right displays 412, 422 is alternated. That is, for a certaintime period the left NIR image is shown in the left display 412, afterthat the display of the left NIR image is stopped and the right NIRimage is shown in the right display 422 for the certain time period. Thetime period may be, for example, just long enough for an average humanto perceive the flickering. A longer time period is also possible. Thisvisualization mode allows to distinguish a specular reflection fromdiffuse reflection and fluorescence. In case of a spot caused by diffusereflection or fluorescence, truly originating from the tissue, the spotwill appear at the same location in both left visualization and rightvisualization, as illustrated in FIG. 4. In case of a spot caused byspecular reflection, however, the spot will appear either in only oneeye, as illustrated in FIG. 5 and FIG. 6, or at different locations inthe left visualization and right visualization, as illustrated in FIG.7. Thus, in case a spot does not occur at the same location in both eyes(flickers and/or “dances”), the user knows there is a specularreflection and move his or her head, and thereby the cameras of thehead-mountable device, to remove the specular reflection from sight.

Another signature of specular reflection versus true tissue contrast orfluorescence is that specular reflections move their position with thehead orientation of the observer whereas true contrast and orfluorescence stays on the same place on the tissue.

The visual effect of specular reflections illustrated in FIGS. 4 to 7may be employed to detect specular reflections in the captured images bymeans of automated image processing. It is observed that specularreflections may be undesirable, because they may obscure the actualsignals, in particular in the invisible wavelength range (e.g.,near-infrared range), the signal that is important is the diffusereflection and/or fluorescence. Specular reflections of the lightemitted by the light source 403 would render the diffuse reflectionsand/or fluorescence invisible. After a left image and a right image havebeen captured, substantially simultaneously, by the left camera 411 andthe right camera 421, with the NIR light source switched on, a specularreflection detection processing may be performed. For example, acomparison between the left image and right image, which have beencaptured simultaneously, may be performed to detect a particularlybright spot. For example, if the intensity at a certain location in animage is above a certain threshold, a bright spot may be detected.Moreover, if such a bright spot is detected in a first one of the leftimage and right image, it may be determined whether a bright spot ispresent in a corresponding location in the other one of the left imageand right image. It may be determined whether a bright spot is presentin the corresponding location, by comparing the image intensity at thecorresponding location to a predetermined threshold, for example.‘Corresponding location’ may mean within a certain distance from thesame location as the bright spot in the first image. This distance maybe determined while taking into account that the corresponding locationsof a visualized item may be slightly different in the left image andright image, according to the disparity of objects in the stereoscopicpair of images.

Alternatively, a known algorithm may be employed to estimate thedisparity, and the corresponding location in the other one of the leftimage and the right image, may be determined based on the estimateddisparity.

For example, if a bright spot is detected in corresponding locations inthe left image and right image according to an estimated disparity, itmay be decided that this is not a specular reflection, but just arelatively intense diffuse reflection, as illustrated in FIG. 4.

For example, in case the bright spot is detected in only one of the leftimage and the right image, but not at the corresponding position in theother one of the left image and the right image, it may be decided thatit is a specular reflection, as shown in FIGS. 5 and 6.

For example, if a bright spot is detected in differing locations justoffset from two corresponding locations in the left image and rightimage according to an estimated disparity, it may be decided that thebright spot is a specular reflection in both images, as shown in FIG. 7.

Preferably, the specular reflection detection is performed for thenon-visible channel (e.g., the NIR channel), because the specularreflections in that channel may not be reduced by the visible lightfilter and/or visible light diaphragm, and it may be vital to properlyvisualize low-intensity features in the non-visible wavelength band.Alternatively, the specular reflection detection may be performed foreach color channel (red, green, blue, non-visible light) separately. Yetalternatively, the intensity of the channels may be combined to detectreflections for all detected wavelength ranges at the same time.

When a specular reflection has been detected, an alarm signal may begenerated to indicate a specular reflection is detected. The alarmsignal may comprise, for example, a sound signal or a visual indication.The visual indication may be shown on the displays 412, 422. Forexample, the detected specular reflection in the non-visible wavelengthrange may be displayed in a different color than the remainder of thenon-visible wavelength image overlay. For example, the non-visiblewavelength may be shown generally as a green overlay on top of a colorimage of the visual wavelength ranges. However, if a spot is identifiedas a specular reflection, the spot may be shown as a darker green as theremainder of the green overlay. This allows the user to move the viewingposition a bit, to a position where there is no specular reflection.

Alternatively, the detected specular reflection may be removed by imageprocessing. For example, the image intensity at the bright spot may belocally reduced by multiplying the intensity values by a reductionfactor, so that the image intensity at the bright spot is made to be ofthe same average level as the average image intensity around the brightspot.

In certain embodiments, neither of the cameras is sensitive to radiationin an invisible wavelength band of radiation. For example, both camerasmay be sensitive to radiation in a visible light wavelength band ofradiation (for example, in case of color cameras, a red wavelengthrange, a green wavelength range, and a blue wavelength range). Thevisible light reduction filter 203 and the visible light diaphragm 204may be omitted as well, as disclosed hereinabove.

The binocular device may optionally comprise a processing unitconfigured to: receive a signal representing a left image from the leftcamera and a right image from the right camera, the left image and theright image being captured substantially simultaneously by the leftcamera and the right camera, respectively; compare the left image to theright image; and detect a specular reflection based on a result of thecomparison.

In an optional particular visualization mode, the binocular device mayalternatingly show a left image based on the left video signalsgenerated by the left camera on the left display and a right image basedon the right video signals generated by the right camera on the rightdisplay, to allow a user to distinguish a specular reflection. Aspecular reflection can be easily identified by comparing the left andright image, as a specular reflection will dance back and forth as theimages are compared whilst fluorescence and diffuse reflections willremain static for both cameras. This visualization mode allows todistinguish a specular reflection from diffuse reflection and/orfluorescence.

The device may detect, suppress, or make known to the user specularreflections within various parts of the optical spectrum.

FIG. 8 shows a view of a combined camera and viewer 800 for one eye(left optics or right optics). In use, the device 800 may be fixed inbetween an eye 810 of the observer and an object, for example a tissue,801, to be observed. In typical use the device 800 may be held orattached close to the eye 810, and at a working distance from the object801 to be observed. The device 800 may comprise input optics including anotch filter 802, a dichroic diaphragm, which may be implemented asshown as a pair 803, 804 of polarization filters, an optics set 805,which may comprise one or more lenses and other optical elements. Theorder of these components may vary depending on the implementation. Thedevice 800 further comprises an image sensor 806, for example a CMOSchip or another type of camera chip. For example, the chip may have asurface that converts radiation projected thereon in both the visiblewavelength band of radiation and the invisible wavelength band ofradiation into an electronic signal. The image sensor 806 iselectrically connected or connectable to electronics 807 for processingthe image signals generated by the image sensor 806. This electronics807 may be incorporated in the binocular device 101, or alternatively,may be implemented in an external processing device, such as a wearablecomputer 102 that has sufficient processing power. This helps to keepthe head-mountable display light-weight.

The device 800 further comprises a micro display 808 for displayingprocessed images based on image signals that are output by theelectronics 807 and transmitted to the micro display 808. The microdisplay may have, for example, a size comparable to an eye. The size ofthe micro display may be arbitrary, as the device 800 further comprisesoutput optics, including ocular display set 809, to project the imageoutput by the micro display 808 onto the retina of the eye 810. Theinput optics 802, 803, 804, 805, and image sensor 806 may be opticallyisolated from the micro display 808 and output optics 809, for exampleby disposing them in two separate compartments with walls that are nottransmissive to radiation of the wavelengths concerned.

In the embodiment shown, the input optics, camera, micro display, andoutput optics are in line, that is, share the same optical central axis811, 812. This way, the user has the impression as if looking straightahead, through e.g. a pair of binoculars. Thus, it is easy for the userto orient himself and his/her hands with respect to the images producedby the micro display 808. In alternative embodiments, there may be aninclination between the central 811 axis of the input optics and camera806 on the one hand, and the central axis 812 of the micro display andoutput optics, on the other hand. For example, the input axis 811 may beinclined a little downwards with respect to the output axis 812.

FIG. 9 shows an example of a timing diagram of the binocular device 400in operation. The timing graphs show time on the horizontal axis, andthe performing of a certain activity on the vertical axes (a highposition means an action is performed). It is observed that the cameramay be configured to capture images at a certain frame rate, asindicated in graph 902.

The light source 403 may have separately controllable visible light andnon-visible light generating capabilities. Alternatively, the lightsource may generate only the non-visible light. In that case, visiblelight may be provided from elsewhere. However, for reasons ofconsistency between the images recorded, it may be preferable to have asingle light source that can generate both visible and non-visible lightin a single light bundle. Moreover, to prevent visible flickering, thevisible light source may be kept continuously emitting radiation whilethe non-visible light is switched on and off alternatingly. Thisprevents flickering not only for the camera images, but also for anyother people in the room that do not have a binocular device 400.

The non-visible light may be generated by the light source instroboscopic fashion. This is illustrated in graphs 901 and 902. Thelight source for non-visible light, such as NIR light, may be configuredto flash at a lower speed than the frame rate, for example at half theframe rate, so that while capturing each first frame the non-visiblelight source is off, and while capturing each second frame, thenon-visible light source is on. One image may be taken with thenon-visible light source switched off. The next image may be taken withthe non-visible light switched on. Thus, two successive images produce aresult of imaging without non-visible light, and a result of imagingwith non-visible light, as indicated at 905. After such a pair of imageshas been captured from each camera, the processing electronics 807 maycalculate an output image based on the pair of captured input images, asindicated at 906 in graph 903. The output image may be displayed by themicro display 808 as soon as the processing in block 906 has beencompleted, as indicated at 907 in graph 904. It will be understood thatthis is only an example timing diagram. Other timings and other order ofsteps may be implemented alternatively.

For example, in the processing step 906, the processing electronics 807may subtract the image with the non-visible light source switched offfrom the image with the non-visible light source switched on. This way,the visible light is subtracted, so that the non-visible light isenhanced in the subtraction image. Thus, if a pixel in the image withthe non-visible light source switched off has a value X, and the samepixel in the image with the non-visible light source switched on has avalue Y, the same pixel in the subtraction image would have the valueY−X. The pixels of the subtraction image that are larger than apredetermined threshold may be blended, in a predetermined visiblecolor, on top of the image that was captured with the non-visible lightsource switched off.

Moreover, a speckle detection mode may be provided, in which the usermay detect speckle by means of a visual effect. For example, thespecular reflections may be indicated by means of an alarm or a visualindication. Alternatively, in the speckle detection mode, illustrated bygraphs 910 and 911, the overlay visualization of the non-visible lightimage is shown alternatingly on only the left display 412, during afirst time interval 911, 912, and on only the right display 422, duringa second time interval 913, 914. In such a case, the viewer can assessif there is a specular reflection in the non-visible domain, byconsidering if there is a spot that appears to be oscillating betweentwo positions. The time intervals 911,912 and 913,914 may be made longeror shorter as desired, for example using a time interval that is atleast as long or longer than the time interval in which two images arecaptured (at 905), to ensure that the flickering is visible to a humanobserver.

FIG. 10 shows a partly worked open view of an exemplary head-mountablebinocular device, viewed from below. FIG. 11 shows a partly worked openview of the same head-mountable binocular device, viewed from the side.In this example, the optical central axis of the left viewing portion issubstantially parallel to the optical central axis of the right viewingportion. Moreover, the optical axis of the left camera portion and theoptical axis of the right camera portion are slightly inclined towardseach other and downwards. However, the alignment of the optical axes isnot a limitation.

Shown in FIGS. 10 and 11 are the camera part comprising the optionalnotch filter 1001, the visible light diaphragm 1003, the camera lens1005, the camera sensor, or image sensor, 1006. Moreover, shown in FIGS.10 and 11 are the display part comprising the micro display 1008, outputoptics 1009, and eye piece 1011, which comprises an annular structuresuitable for holding close to the eye 1010. Moreover, shown in FIGS. 10and 11 is the light source 1012. These items have been described ingreater detail hereinabove, and therefore their properties are notrepeated here.

FIG. 12 shows how the head-mountable device can be attached to the headof a user, by means of at least one strip 1201 connected to the supportstructure 1204 that can be fitted around the head. The eyepieces may bealigned to the eyes of the user, as shown.

FIG. 13 shows several wavelength bands of visible light and invisiblelight (ultraviolet and near-infrared), in nanometers (nm). It is notedthat blue may be centered around 445 a nanometer wavelength, green maybe centered around a 535 nanometers wavelength, and red may be centeredaround 575 nanometers wavelength. Ultraviolet may be considered to beradiation with a wavelength below around 380 nanometers. Near-infraredmay be considered to be radiation in a wavelength range from about 740nanometers up to about 1000 nanometers. Infrared radiation goes furtherbeyond 1000 nanometers. It is noted that these wavelengths are providedpurely as illustrative examples. The devices described herein may bedesigned for different wavelengths for detection, processing, andvisualization.

FIG. 14 shows a flowchart of a method of visualizing visible andinvisible radiation. The method comprises, in step 1401, receivingradiation, about a same field of view along respective left and rightinput optical axes, by left and a right input optics coupled to asupport structure, and transmitting the light onto a left image sensorof a left camera and a right image sensor of a right camera,respectively, the left camera and the right camera being coupled to thesupport structure, wherein at least one of the cameras is sensitive toboth radiation in an invisible wavelength band of radiation andradiation in a visible light wavelength band of radiation, wherein theinput optics of the camera that is sensitive to the invisible wavelengthband of radiation is transmissive for the invisible wavelength band ofradiation and reductive for the visible light wavelength band ofradiation. The method further comprises, in step 1402, creating left andright video signals from the detected radiation received by the leftcamera and the right camera, respectively. The method further comprises,in step 1403, presenting, by a left display and a right display coupledto the support structure and to be viewed by a pair of eyes of a userthrough a left eyepiece operatively connected to the left display and aright eyepiece operatively connected to the right display, left andright video images formed with visible light based respectively on theleft and right video signals.

It may be observed that the features for reduction of specularreflections may be implemented in a binocular device, even in absence ofthe reduction of visible light. That is, the visible light filter 202and the visible light diaphragm 204 may both be omitted in a device ormethod that includes the image processing functionality of detecting aspecular reflection.

Certain embodiments comprise a head-mountable device, or a binoculardevice that is designed to be held directly in front of the eyes, whichdevice has at least one camera and/or light source, to observe theobject from a working distance in an open space (e.g., an indoor oroutdoor environment). In contrast, other applications, such asendoscopy, may operate in a largely dark cavity in which lighting can becontrolled freely. This open space poses some constraints on thelighting. First, there is the presence of environmental light caused byexternal light sources. Second, it may not be possible to optimize thelighting conditions purely for the cameras of the binocular device,because other people and/or other camera equipment should preferably notbe disturbed by the lighting caused by the binocular device. Thetechniques disclosed herein may help to improve the usability of thebinocular device under these circumstances. For example, the cameras ofthe binocular device may be equipped with a high dynamic range, highquality optics, special image processing techniques, and/or polarizingfilters, dichroic filters, polarizing diaphragms, and/or dichroicdiaphragms, as described herein. For this reason also, the light sourceof the binocular device may be configured to keep the emitted light inthe visible wavelength band as much as possible constant, not to disturbany people around. As described above, the emitted light in theinvisible wavelength band may be flashing in stroboscopic fashion, sothat images of visible light can be combined with images with invisiblelight. Since the flashing occurs in the invisible wavelength band, thisdoes not disturb the people around.

Although several techniques have been disclosed hereinabove in relationto a head-mountable binocular device, this is not a limitation. It isobserved that the features of a image processing techniques, and/orpolarizing filters, dichroic filters, polarizing diaphragms, and/ordichroic diaphragms, may also be applied to a camera in general.

According to another aspect, a camera comprises input optics and animage sensor, the image sensor being configured to create a video signalfrom detected optical radiation received from the input optics about afield of view along an input optical axis, at least one of the camerasbeing sensitive to both radiation in an invisible wavelength band ofradiation and radiation in a visible light wavelength band of radiation,the input optics being transmissive for the invisible wavelength bandand reductive for the visible light wavelength band. Such a camera maybe designed for many different uses, such as, for example, endoscopy.

Optionally a display is configured to present a video image, formed withvisible light by the display, based on the video signal.

Optionally, the display is arranged to be viewed by a user through aneyepiece operatively connected to the display.

The input optics of the camera may comprise a polarizing filtercomprising at least one layer of a polarizing material.

The polarizing filter may comprise at least two layers of the polarizingmaterial, having a mutually orthogonal polarization direction.

The device may comprise a light source coupled to the camera by asupport structure, the light source being capable of generatingradiation within at least the invisible wavelength band and the visiblelight wavelength band, wherein the light source is configured togenerate beams of emitted visible light and invisible radiation that arealigned to be substantially identical in geometrical shape and position.

The device may comprise a light source coupled to the support structure,capable of generating radiation within at least the invisible wavelengthband and the visible light wavelength band, wherein the light sourcefurther comprises a polarizing filter configured to polarize the visiblelight within the visible light wavelength band output by the lightsource and transmit the radiation in the invisible wavelength band,wherein a polarization direction of the polarizing filter of the lightsource is substantially orthogonal to a polarization direction of thepolarizing filter of the input optics.

The input optics may comprise a diaphragm having an aperture, thediaphragm around the aperture being reductive for light in the visiblelight wavelength band, while the diaphragm is transmissive for the lightin the infrared wavelength band.

The input optics may comprise a lens with autofocus, wherein theautofocus is configured to bring into focus the radiation in theinvisible wavelength band.

The input optics of the camera may comprise a filter that is reductivefor the light in the visible light wavelength band in addition to thediaphragm.

The input optics of the camera may comprise a filter comprising iodinefor selectively reducing the radiation in the visible light wavelengthband.

The device may comprise a light source coupled to the camera by asupport structure, for generating at least invisible light and visiblelight, wherein the light source is configured to intermittently emit theinvisible light while keeping the visible light intensity substantiallyconstant, wherein the camera is configured to capture at least one imagewith the emitted invisible light and at least one image without theemitted invisible light.

The device may comprise a processing unit configured to calculate anenhanced invisible-light image based on the captured image with theemitted invisible light and the captured image without the emittedinvisible light.

The image sensor may comprise a sensor die that is sensitive to both theinfrared wavelength band of radiation and the visible wavelength band ofradiation, wherein the sensor die is configured to output the videosignal corresponding to both the radiation in the infrared wavelengthband and the radiation in the visible wavelength band.

Some or all aspects of the invention may be suitable for beingimplemented in form of software, in particular a computer programproduct. The computer program product may comprise a computer programstored on a non-transitory computer-readable media. Also, the computerprogram may be represented by a signal, such as an optic signal or anelectro-magnetic signal, carried by a transmission medium such as anoptic fiber cable or the air. The computer program may partly orentirely have the form of source code, object code, or pseudo code,suitable for being executed by a computer system. For example, the codemay be executable by one or more processors.

The examples and embodiments described herein serve to illustrate ratherthan limit the invention. The person skilled in the art will be able todesign alternative embodiments without departing from the spirit andscope of the present disclosure, as defined by the appended claims andtheir equivalents. Reference signs placed in parentheses in the claimsshall not be interpreted to limit the scope of the claims. Itemsdescribed as separate entities in the claims or the description may beimplemented as a single hardware or software item combining the featuresof the items described.

Certain aspects are defined in the following clauses.

Clause 1. A binocular device for visualizing visible and invisibleradiation, comprising

-   -   a support structure;    -   a left camera and a right camera coupled to the support        structure, the left camera comprising left optics and a left        image sensor, the right camera comprising right optics and a        right image sensor;    -   the left image sensor and the right image sensor being        configured to create left and right video signals from detected        optical radiation received from the corresponding left and right        input optics about a same field of view along respective left        and right input optical axes,    -   at least one of the cameras being sensitive to both radiation in        an invisible wavelength band of radiation and radiation in a        visible light wavelength band of radiation, the input optics of        said at least one of the cameras being transmissive for the        invisible wavelength band and reductive for the visible light        wavelength band.

Clause 2. The binocular device of clause 1, further comprising

-   -   a left display and a right display coupled to the support        structure and arranged to be viewed by a pair of eyes of a user        through a left eyepiece operatively connected to the left        display and a right eyepiece operatively connected to the right        display, wherein the left display and the right display are        configured to present left and right video images, formed with        visible light by the left display and the right display, based        respectively on the left and right video signals.

Clause 3. The binocular device of any preceding clause, wherein theinput optics of at least the camera that is sensitive to radiation inthe invisible wavelength band comprises a polarizing filter comprisingat least one layer of a polarizing material.

Clause 4. The binocular device of clause 3, wherein the polarizingfilter comprises at least two layers of the polarizing material, havinga mutually orthogonal polarization direction.

Clause 5. The binocular device of any preceding clause, furthercomprising a light source coupled to the support structure, capable ofgenerating radiation within at least the invisible wavelength band andthe visible light wavelength band, wherein the light source isconfigured to generate beams of emitted visible light and invisibleradiation that are aligned to be substantially identical in geometricalshape and position.

Clause 6. The binocular device of clause 3, further comprising

-   -   a light source coupled to the support structure, capable of        generating radiation within at least the invisible wavelength        band and the visible light wavelength band, wherein the light        source further comprises a polarizing filter configured to        polarize the visible light within the visible light wavelength        band output by the light source and transmit the radiation in        the invisible wavelength band;    -   wherein a polarization direction of the polarizing filter of the        light source is substantially orthogonal to a polarization        direction of the polarizing filter of the input optics.

Clause 7. The binocular device of any preceding clause, wherein theinput optics corresponding to the camera that is sensitive to invisibleradiation comprises a diaphragm having an aperture, the diaphragm aroundthe aperture being reductive for light in the visible light wavelengthband, while the diaphragm is transmissive for the light in the invisiblewavelength band.

Clause 8. The binocular device of clause 7, wherein the input opticscomprises a lens with autofocus, wherein the autofocus is configured tobring into focus the radiation in the invisible wavelength band.

Clause 9. The binocular device of clause 7, wherein the input optics ofthe camera that is sensitive to radiation in the invisible wavelengthband further comprises a filter that is reductive for the light in thevisible light wavelength band in addition to the diaphragm.

Clause 10. The binocular device of any preceding clause, wherein theinput optics of the camera that is sensitive to radiation of theinvisible wavelength band comprises a filter comprising iodine forselectively reducing the radiation in the visible light wavelength band.

Clause 11. The binocular device of any preceding clause, furthercomprising

a processing unit configured to:receive a signal representing a left image from the left camera and aright image from the right camera, the left image and the right imagebeing captured substantially simultaneously by the left camera and theright camera, respectively;

-   -   compare the left image to the right image; and    -   detect a specular reflection based on a result of the        comparison.

Clause 12. The binocular device of any preceding clause, furthercomprising

-   -   a light source coupled to the support structure, for generating        at least radiation in the invisible wavelength band and visible        light, wherein the light source is configured to intermittently        emit the invisible light while keeping the visible light        intensity substantially constant, wherein the camera is        configured to capture at least one image with the emitted        invisible light and at least one image without the emitted        invisible light.

Clause 13. The binocular device of clause 12, further comprising aprocessing unit configured to calculate an enhanced image of radiationin the invisible wavelength band based on the captured image with theemitted invisible light and the captured image without the emittedinvisible light.

Clause 14. The binocular device of any preceding clause, wherein each ofthe left camera and the right camera is sensitive to radiation in theinvisible wavelength band and radiation in the visible light wavelengthband, each of the left input optics and the right input optics beingtransmissive for the invisible wavelength band and reductive for thevisible light wavelength band.

Clause 15. The binocular device of any preceding clause, wherein theimage sensor of the camera that is sensitive to the radiation in theinvisible wavelength band comprises a sensor die that is sensitive toboth the invisible wavelength band of radiation and the visiblewavelength band of radiation, wherein the sensor die is configured tooutput the video signal corresponding to both the radiation in theinvisible wavelength band and the radiation in the visible wavelengthband.

Clause 16. The binocular device of any preceding clause, wherein theinvisible wavelength band of radiation is a near-infrared wavelengthband of radiation.

Clause 17. A method of visualizing visible and invisible radiation,comprising

-   -   receiving radiation, about a same field of view along respective        left and right input optical axes, by left and a right input        optics coupled to a support structure, and transmitting the        light onto a left image sensor of a left camera and a right        image sensor of a right camera, respectively, the left camera        and the right camera being coupled to the support structure,    -   wherein at least one of the cameras is sensitive to both        radiation in an invisible wavelength band of radiation and        radiation in a visible light wavelength band of radiation,        wherein the input optics of the camera that is sensitive to the        invisible wavelength band of radiation is transmissive for the        invisible wavelength band of radiation and reductive for the        visible light wavelength band of radiation; and    -   creating left and right video signals from the detected        radiation received by the left camera and the right camera,        respectively.

Clause 18. The method of clause 17, further comprising

-   -   presenting, by a left display and a right display coupled to the        support structure and to be viewed by a pair of eyes of a user        through a left eyepiece operatively connected to the left        display and a right eyepiece operatively connected to the right        display, left and right video images formed with visible light        based respectively on the left and right video signals.

1. A binocular device for visualizing optical radiation, comprising asupport structure; a left camera and a right camera coupled to thesupport structure, the left camera comprising left optics and a leftimage sensor, the right camera comprising right optics and a right imagesensor, the left image sensor and the right image sensor beingconfigured to create left and right video signals from detected opticalradiation received from the corresponding left and right input opticsabout a same field of view along respective left and right input opticalaxes; the device further comprising a processing unit configured to:receive a signal representing a left image from the left camera and aright image from the right camera, the left image and the right imagebeing captured substantially simultaneously by the left camera and theright camera, respectively; compare the left image to the right image;and detect a specular reflection based on a result of the comparison. 2.The binocular device of claim 1, further comprising a left display and aright display coupled to the support structure and arranged to be viewedby a pair of eyes of a user through a left eyepiece operativelyconnected to the left display and a right eyepiece operatively connectedto the right display, wherein the left display and the right display areconfigured to present left and right video images, formed with visiblelight by the left display and the right display, based respectively onthe left and right video signals.
 3. The binocular device of claim 1,wherein at least one of the cameras is sensitive to both radiation in aninvisible wavelength band of radiation and radiation in a visible lightwavelength band of radiation, wherein the input optics of said at leastone of the cameras is transmissive for the invisible wavelength band andreductive for the visible light wavelength band.
 4. The binocular deviceof claim 3, wherein the input optics of at least the camera that issensitive to radiation in the invisible wavelength band comprises apolarizing filter comprising at least one layer of a polarizingmaterial.
 5. The binocular device of claim 4, wherein the polarizingfilter comprises at least two layers of the polarizing material, the twolayers of the polarizing material having a mutually orthogonalpolarization direction.
 6. The binocular device of claim 3, furthercomprising a light source coupled to the support structure, capable ofgenerating radiation within at least the invisible wavelength band andthe visible light wavelength band, wherein the light source isconfigured to generate beams of emitted visible light and invisibleradiation that are aligned to be substantially identical in geometricalshape and position.
 7. The binocular device of claim 4, furthercomprising a light source coupled to the support structure, capable ofgenerating radiation within at least the invisible wavelength band andthe visible light wavelength band, wherein the light source furthercomprises a polarizing filter configured to polarize the visible lightwithin the visible light wavelength band output by the light source andtransmit the radiation in the invisible wavelength band; wherein apolarization direction of the polarizing filter of the light source issubstantially orthogonal to a polarization direction of the polarizingfilter of the input optics.
 8. The binocular device of claim 3, whereinthe input optics corresponding to the camera that is sensitive toinvisible radiation comprises a diaphragm having an aperture, thediaphragm around the aperture being reductive for light in the visiblelight wavelength band, while the diaphragm is transmissive for the lightin the invisible wavelength band.
 9. The binocular device of claim 8,wherein the input optics comprises a lens with autofocus, wherein theautofocus is configured to bring into focus the radiation in theinvisible wavelength band.
 10. The binocular device of claim 8, whereinthe input optics of the camera that is sensitive to radiation in theinvisible wavelength band further comprises a filter that is reductivefor the light in the visible light wavelength band in addition to thediaphragm.
 11. The binocular device of claim 3, wherein the input opticsof the camera that is sensitive to radiation of the invisible wavelengthband comprises a filter comprising iodine for selectively reducing theradiation in the visible light wavelength band.
 12. The binocular deviceof claim 1, further comprising a light source coupled to the supportstructure, for generating at least radiation in the invisible wavelengthband and visible light, wherein the light source is configured tointermittently emit the invisible light while keeping the visible lightintensity substantially constant, wherein the camera is configured tocapture at least one image with the emitted invisible light and at leastone image without the emitted invisible light.
 13. The binocular deviceof claim 12, further comprising a processing unit configured tocalculate an enhanced image of radiation in the invisible wavelengthband based on the captured image with the emitted invisible light andthe captured image without the emitted invisible light.
 14. Thebinocular device of claim 1, wherein each of the left camera and theright camera is sensitive to radiation in the invisible wavelength bandand radiation in the visible light wavelength band, each of the leftinput optics and the right input optics being transmissive for theinvisible wavelength band and reductive for the visible light wavelengthband.
 15. The binocular device of claim 1, wherein the image sensor ofthe camera that is sensitive to the radiation in the invisiblewavelength band comprises a sensor die that is sensitive to both theinvisible wavelength band of radiation and the visible wavelength bandof radiation, wherein the sensor die is configured to output the videosignal corresponding to both the radiation in the invisible wavelengthband and the radiation in the visible wavelength band.
 16. The binoculardevice of claim 1, wherein the invisible wavelength band of radiation isa near-infrared wavelength band of radiation.
 17. A method ofvisualizing optic radiation, comprising receiving radiation, about asame field of view along respective left and right input optical axes,by left and a right input optics coupled to a support structure, andtransmitting the light onto a left image sensor of a left camera and aright image sensor of a right camera, respectively, the left camera andthe right camera being coupled to the support structure; and creatingleft and right video signals from the detected radiation received by theleft camera and the right camera, respectively; receiving, by aprocessor, a signal representing a left image from the left camera and aright image from the right camera, the left image and the right imagebeing captured substantially simultaneously by the left camera and theright camera, respectively; comparing, by the processor, the left imageto the right image; and detecting, by the processor, a specularreflection based on a result of the comparison.
 18. The method of claim17, further comprising presenting, by a left display and a right displaycoupled to the support structure and to be viewed by a pair of eyes of auser through a left eyepiece operatively connected to the left displayand a right eyepiece operatively connected to the right display, leftand right video images formed with visible light based respectively onthe left and right video signals.
 19. A binocular device for visualizingoptical radiation, comprising a support structure; a left camera and aright camera coupled to the support structure, the left cameracomprising left optics and a left image sensor, the right cameracomprising right optics and a right image sensor, the left image sensorand the right image sensor being configured to create left and rightvideo signals from detected optical radiation received from thecorresponding left and right input optics about a same field of viewalong respective left and right input optical axes; a left display and aright display coupled to the support structure and arranged to be viewedby a pair of eyes of a user through a left eyepiece operativelyconnected to the left display and a right eyepiece operatively connectedto the right display, wherein the left display and the right display areconfigured to present left and right video images, formed with visiblelight by the left display and the right display, based respectively onthe left and right video signals; wherein the binocular device isconfigured to, in a particular visualization mode, alternatingly show aleft image based on the left video signals on the left display and aright image based on the right video signals on the right display, toallow a user to distinguish a specular reflection.
 20. (canceled)